fix(rules): prove ContainerCpuThrottled can fire, and say that it cannot (#185) - #318

Closes #185, taking the path the issue argues for rather than the one that would make the rule live.

Not adding a CPU quota

The tempting fix is the wrong one.

Agreed, and the reasoning is now in the rule itself rather than only in the issue: the stack peaks at ~0.4 of 4 cores, HostHighLoad already fires at load15 > 2 x cores, and a quota tight enough to matter would be saturated inside individual 100ms CFS windows during a scrape burst — manufacturing alert fatigue to justify a control that prevents nothing.

The rule says what it is

A comment stating it is inert, why a quota is the wrong answer, and exactly what would make it live: any service gaining cpus: or deploy.resources.limits.cpus. Nothing else. It is kept against the day a quota is set for some other reason — a noisy-neighbour guest, a service that genuinely needs bounding.

The tests are what make keeping it safe

Per the convention in containers.test.yaml's header, a firing case paired with a quiet one:

• Fires — 600 CFS periods a minute (the default 100ms period), 180 of them throttled: 30% against a 0.25 threshold. Realistic rather than a contrived 100%.
• Quiet — both counters flat at 0, which is exactly what cAdvisor reports for every container in this estate and will keep reporting until something sets a quota. clamp_min holds the denominator at 1, so this evaluates to 0 rather than NaN.

Mutation-tested rather than trusted. A green suite proves nothing on its own — that is the #63 lesson. Both of these fail it:

One correction from writing them: I had the for: arithmetic wrong, assuming rate() waits for its 15m window to fill. It returns a value from the second sample, so the alert is pending at 1m and fires at 31m, not 45m. promtool caught it, and the paired 25m case now asserts the for: rather than assuming it.

Two knock-ons

• docs/observability.md's coverage counts moved — 27→28 tested, 29→28 not. check_docs.py caught it, which is the docs/observability.md says three receivers; there are four #212/Five places say seven VLANs; the firewall has six, and this repo's own tables already list six #209 machinery working on a change unrelated to either.
• That paragraph carried advice Five places say seven VLANs; the firewall has six, and this repo's own tables already list six #209 had already invalidated: "Keep each count on one line: the checker reads prose line by line, so a phrase wrapped mid-claim is a claim it cannot see." Untrue since the checker went whole-file. Removed. Nothing would have flagged it — I only saw it because it sat beside the count I was editing.

Checks

validate.sh — 48 PASS, 0 FAIL, 2 SKIP. promtool test rules now covers 9 files in observability, up from 8.

🤖 Generated with Claude Code